Many significant and commercially important uses of modern computer technology relate to images. These include image processing, image analysis and computer vision applications. A challenge in the utilization of computers to accurately and correctly perform operations relating to images is the development of algorithms that truly reflect and represent physical phenomena occurring in the visual world. For example, the ability of a computer to correctly and accurately distinguish between a shadow and a material object edge within an image has been a persistent challenge to scientists. Edge detection is a fundamental task in image processing because without accurate and correct detection of the edges of physical objects, no other processing of the image is possible. If a cast shadow is indistinguishable from the object casting the shadow, it would not be possible for the computer to recognize the object.
An early and conventional approach to object edge detection involves an analysis of brightness boundaries in an image. In the analysis it is assumed that a boundary caused by a material object will be sharp, while a boundary caused by a shadow will be soft or gradual due to the penumbra effect of shadows. While this approach can be implemented by algorithms that can be accurately executed by a computer, the results will often be incorrect. In the real world there are many instances wherein shadows form sharp boundaries, and conversely, material object edges form soft boundaries. Thus, when utilizing conventional techniques for shadow and edge recognition, there are significant possibilities for false positives and false negatives for shadow recognition. That is, for example, a material edge that imitates a shadow and is thus identified incorrectly by a computer as a shadow or a sharp shadow boundary that is incorrectly interpreted as an object boundary.
Once shadows and object edges are identified, a typical computerized operation is manipulation of the image to, for example, remove shadows from the image. Most scenes depicted in an image have a dominant illuminant, defined as a direct or incident illuminant. The incident illuminant causes shadows. The component of radiance onto a surface in the scene that is not from the incident illuminant, is referred to as an indirect or ambient illuminant. It is the ambient illuminant that is present within a shadow. While much of the energy of the ambient illuminant may come from the incident illuminant, it has generally interacted with the environment.
Typical solutions for manipulating images focus on the incident illuminant. Models have been developed for computerized image pixel manipulation based upon the assumption that the ambient illumination has the same spectral characteristics as the incident illumination or is non-existent. One such known solution is the dichromatic reflection model, which describes the variations in appearance caused by the combination of body and surface reflection on a material. Body reflection is what is normally considered the color of the material. The surface reflection is referred to as a highlight or specularity of the material reflecting the illuminant. The known dichromatic reflection model assumes a single incident illuminant and does not account for a non-zero ambient illuminant. Thus, results of image manipulation based upon the dichromatic reflection model are often not color correct.
Other useful solutions include color spaces such as hue and chromaticity, and other normalized color spaces that attempt to factor out the effect of changing the intensity of the incident illuminant on the intensity of the reflection from a surface. However, these color models have met with limited success in solving practical vision problems. Consequently, there is a growing realization that simple models of illumination do not correctly reflect the visual world, and thus, do not provide color correct manipulations. Recent research has focused upon empirical models of color change over a single material. This approach is not restricted by any prior assumptions about the behavior of illumination color or reflectance.
There is also significant amount of research attempting to determine the complete illumination environment. These methods are based upon multiple images of the same scene and/or knowledge of the scene geometry. In one research project, the existence and importance of complex indirect illumination is acknowledged. However, the method requires both a known geometry of a scene and two images. While these research efforts approach a solution that can extract and represent illumination environments of significant complexity, they cannot be used in environments such as, for example, consumer photography, or with existing photos or in any situation where either taking multiple images of a scene from different points of view or inserting objects into a scene are not readily possible or are unreasonable.